Prior Gradient Mask Guided Pruning-Aware Fine-Tuning

We proposed a Prior Gradient Mask Guided Pruning-aware Fine-Tuning (PGMPF) framework to accelerate deep Convolutional Neural Networks (CNNs). In detail, the PGMPF selectively suppresses gradient of those ”unimportant” parameters via prior mask generated by pruning criterion during fine-tuning. has three charming characteristics over previous works: (1) network A typical pipeline consists training, and fine-tuning, which are relatively independent, while utilizes variant as guide without complicated criteria. (2) An excellent tradeoff between large model capacity fine-tuning stable convergence speed obtain final compact model. Previous works preserve more training information pruned pursue better performance, would incur catastrophic non-convergence for rates, our greatly stabilizes phase gradually constraining learning rate parameters. (3) Channel-wise random dropout impose some noise further improve robustness Experimental results on image classification benchmarks CIFAR10/ 100 ILSVRC-2012 demonstrate effectiveness method various CNN architectures, datasets rates. Notably, ILSVRC-2012, reduces 53.5% FLOPs ResNet-50 with only 0.90% top-1 accuracy drop 0.52% top-5 drop, advanced state-of-the-art negligible extra computational cost.